Replication is one technique utilized to minimize data loss and improve the availability of data. During replication, a copy of the same data is stored at each of several sites or nodes. If the working copy (the copy currently being used by an application) of that data is lost, one of the replicated copies can be used.
Performance metrics used to assess replication systems include recovery point objective (RPO) and recovery time objective (RTO). The RPO metric is used to indicate the point in time to which data (e.g., application data, system state, and the like) must be recovered by a replication system. In other words, RPO indicates how much data loss (e.g., 2 hours worth of data loss) can be tolerated by applications associated with the replication system. The RTO metric is used to indicate the time within which systems, applications, and/or operations associated with the replication system must be recovered.
Optimally, a replication system would provide for instantaneous (zero RTO) and complete (zero RPO) recovery of data from one or more remote sites at a great distance from the data-generating primary node. However, implementation of such a replication system using conventional techniques would be extremely inefficient, in terms of both write operation latency and cost. In particular, the cost of the high-speed link(s) required by such replication systems has discouraged their implementation however in all but a small number of application environments.
Replication systems in which high-frequency data replication is performed over short, high-speed links, as well as systems in which low-frequency data replication is performed over longer, low-speed links, similarly suffer from a number of drawbacks (e.g., a poor RPO metric, high write operation/application latency, high cost, and the like). Consequently, a number of replication systems have been implemented in which such short-distance, high-speed/frequency replication (e.g., real-time or synchronous replication) is coupled (e.g., cascaded) with long-distance, low-speed/frequency replication. In a cascaded replication system, copies of all the data generated and/or stored at the primary node are maintained at both an intermediate node (e.g., via short-distance, high-speed/frequency replication between the primary and intermediary nodes) and a remote node (e.g., via long-distance, low-speed/frequency replication between the intermediate and remote nodes).
In a cascaded replication system, updates to data stored at the primary node are typically replicated synchronously to the intermediate node from the primary node. The updates are then replicated asynchronously from the intermediate node to the remote node. Because the asynchronous replication is controlled by the intermediate node, the primary node has no information indicating the state of data stored at the remote node, relative to data stored at the primary node. If the intermediate node fails, the primary node will not be able to determine how far behind the data at the remote node is, relative to the data at the primary node, and thus the primary node will be unable to continue ongoing replication to the remote node. Instead, the primary node will have to reinitialize the data at the remote node to a known state and then restart replication. As this example shows, improved techniques for dealing with intermediate node failures are desirable.